We introduce a new class of generative models for music called live music models that produce a continuous stream of music in real-time with synchronized user control. We release Magenta RealTime, an open-weights live music model that can be steered using text or audio prompts to control acoustic style. On automatic metrics of music quality, Magenta RealTime outperforms other open-weights music generation models, despite using fewer parameters and offering first-of-its-kind live generation capabilities. We also release Lyria RealTime, an API-based model with extended controls, offering access to our most powerful model with wide prompt coverage. These models demonstrate a new paradigm for AI-assisted music creation that emphasizes human-in-the-loop interaction for live music performance.

Adam Roberts' Lake of Darkness opens as two space ships investigate a black hole The starting point for this novel was that I wanted to write utopian fiction. I hadn't done this before: all my previous novels have been straight science fiction. But utopia, the genre that imagines a better, or a perfect, world, is older than science fiction: the first utopian novel, the work that coined the term, was written by Thomas More all the way back in 1516. I was interested in what happened to the mode: More's Utopia generated lots of imitators. Through the 17th and 18th centuries, a great many utopian books, novels, tracts and treatises were written.

Synthesizing performing guitar sound is a highly challenging task due to the polyphony and high variability in expression. Recently, deep generative models have shown promising results in synthesizing expressive polyphonic instrument sounds from music scores, often using a generic MIDI input. In this work, we propose an expressive acoustic guitar sound synthesis model with a customized input representation to the instrument, which we call guitarroll. We implement the proposed approach using diffusion-based outpainting which can generate audio with long-term consistency. To overcome the lack of MIDI/audio-paired datasets, we used not only an existing guitar dataset but also collected data from a high quality sample-based guitar synthesizer. Through quantitative and qualitative evaluations, we show that our proposed model has higher audio quality than the baseline model and generates more realistic timbre sounds than the previous leading work.

State-of-the-art language models like T5 have revolutionized the NLP landscape, but their computational demands hinder a large portion of the research community. To address this challenge, we present nanoT5, a specially-optimized PyTorch framework for efficient pre-training and fine-tuning of T5 models. Drawing on insights from optimizer differences and prioritizing efficiency, nanoT5 allows a T5-Base model to be pre-trained on a single GPU in just 16 hours, without any loss in performance. With the introduction of this open-source framework, we hope to widen the accessibility to language modelling research and cater to the community's demand for more user-friendly T5 (Encoder-Decoder) implementations. We make our contributions, including configurations, codebase, pre-training insights, and pre-trained models, available to the public.

Pretrained multilingual large language models have typically used heuristic temperature-based sampling to balance between different languages. However previous work has not systematically evaluated the efficacy of different pretraining language distributions across model scales. In this paper, we propose a new sampling method, UniMax, that delivers more uniform coverage of head languages while mitigating overfitting on tail languages by explicitly capping the number of repeats over each language's corpus. We perform an extensive series of ablations testing a range of sampling strategies on a suite of multilingual benchmarks, while varying model scale. We find that UniMax outperforms standard temperature-based sampling, and the benefits persist as scale increases. As part of our contribution, we release: (i) an improved and refreshed mC4 multilingual corpus consisting of 29 trillion characters across 107 languages, and (ii) a suite of pretrained umT5 model checkpoints trained with UniMax sampling.

DURHAM – Google Brain's Magenta project, which is exploring the creative potential of machine learning (ML) and artificial intelligence (AI), has developed considerably since Google announced it at Moogfest three years ago. And, Magenta makes many of its ongoing developments available publicly online and collects feedback from musicians, artists and other users to advance the project. Adam Roberts, senior software engineer and ML researcher discussed the nuts and bolts of Magenta at Moogfest over the weekend. Roberts, who did undergraduate work at the University of North Carolina at Chapel Hill, earned his PhD at Berkley, California, where he studied machine learning applied to genomics. Google is developing both hardware and software to explore the potential of machine learning via its Magenta research, Roberts said.

JUDY WOODRUFF: The idea, how technology, music and science can inspire one another, and to the creation of distinct new sounds. Jeffrey Brown is back to take us to an unusual gathering held just a few days ago in Durham, North Carolina. JEFFREY BROWN: Start with a circuit board, add knobs and dials, solder everything together, and, eventually, if you know what you're doing, you have an instrument that can do this. Moogfest, named after inventor Robert Moog, is a celebration of the art, engineering and technology of synthesizers, machines that create sounds electronically. By night, it's a festival of different genres of music, centered on, as they call them here, synths.